Papers with cross-lingual models
Transferability of Syntax-Aware Graph Neural Networks in Zero-Shot Cross-Lingual Semantic Role Labeling (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies in cross-lingual semantic role labeling (SRL) lack a comprehensive analysis of their network selection. |
| Approach: | They compare the transferability of graph neural network-based models with universal dependency trees to English and 23 target languages. |
| Outcome: | The proposed models perform better in resource-poor languages than in resource rich ones. |
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for pretraining cross-lingual models are limited in their size due to the limited amount of parallel corpora. |
| Approach: | They propose a method that encourages the model to align multiple languages with monolingual corpora to overcome the constraint of the parallel corpus size. |
| Outcome: | The proposed method outperforms existing cross-lingual models and delivers new state-of-the-art results in various cross-linguistic downstream tasks. |
Cross-lingual Few-Shot Learning on Unseen Languages (2022.aacl-main)
Copied to clipboard
| Challenge: | Large pre-trained language models have demonstrated the ability to obtain good performance on downstream tasks with limited examples in resource-rich languages. |
| Approach: | They propose to use a downstream sentiment analysis task to analyze the effectiveness of several few-shot learning strategies across 12 languages, including 8 unseen languages, to compare results. |
| Outcome: | The proposed model, XLM-R, gives the best performance on a task with few examples in resource-rich languages. |
Exposing the limits of Zero-shot Cross-lingual Hate Speech Detection (2021.acl-short)
Copied to clipboard
| Challenge: | a lack of labeled, non-English resources for hate speech detection limits research on hate speech . a recent study shows that zero-shot, cross-lingual learning models cannot be used as they are . lack of consistency limits research, and lack of models for non-english languages limits learning . |
| Approach: | They propose a zero-shot, cross-lingual transfer learning framework for hate speech detection . they use benchmark data sets in English, Italian, and Spanish to detect hate speech . |
| Outcome: | The proposed framework can't be used as it is, but needs to be carefully designed, the authors say . they find that non-hateful, language-specific taboo interjections are misinterpreted as signals of hate speech . |
Allocating Large Vocabulary Capacity for Cross-Lingual Language Model Pre-Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models require a more expressive vocabulary to represent all languages . however, increasing the vocabulary size significantly slows down the pre-training speed . |
| Approach: | They propose an algorithm VoCap to determine the desired vocabulary capacity of each language. |
| Outcome: | The proposed algorithm improves cross-lingual model pre-training while reducing side effects of increasing vocabulary size. |
Cognition-aware Cognate Detection (2021.eacl-main)
Copied to clipboard
Diptesh Kanojia, Prashant Sharma, Sayali Ghodekar, Pushpak Bhattacharyya, Gholamreza Haffari, Malhar Kulkarni
| Challenge: | Existing approaches to cognate detection use orthographic, phonetic and semantic similarity based features sets. |
| Approach: | They propose a method for enriching feature sets with cognitive features extracted from gaze behaviour data from human readers’ gaze behaviour. |
| Outcome: | The proposed method improves cognate detection performance by 10% and 12% over existing methods. |
VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work in multilingual pretraining relies on the shared vocabulary and bilingual contexts to encourage the correlation across languages. |
| Approach: | They propose to plug a cross-attention module into a Transformer encoder to explicitly build the interdependence between languages. |
| Outcome: | The proposed model outperforms existing models on XTREME and English-to-French translation datasets. |
GLUECoS: An Evaluation Benchmark for Code-Switched NLP (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show multilingual contextual embedding models perform better on cross-lingual and multilingual tasks. |
| Approach: | They propose to evaluate multilingual contextual embedding models on multilingual data . they use language identification from text, POS tagging, Named Entity Recognition and Question Answering . |
| Outcome: | The proposed benchmark evaluates models on language identification from text, POS tagging, Named Entity Recognition, Question Answering and a new task for code-switching, Natural Language Inference. |
Evaluating Sub-word Embeddings in Cross-lingual Models (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches to learning sub-word embeddings for out-of-vocabulary words have not considered sub- word embedds in cross-lingual models. |
| Approach: | They propose to use sub-word embeddings to form cross-lingual embeddables for out-of-vocabulary (OOV) words for which no embeddibles are available. |
| Outcome: | The proposed bilingual lexicon induction task shows that sub-word embeddings can be leveraged to form cross-lingual embeddables for OOV words. |
Investigating Transfer Learning in Multilingual Pre-trained Language Models through Chinese Natural Language Inference (2021.findings-acl)
Copied to clipboard
| Challenge: | Multilingual transformers have been shown to have remarkable transfer skills in zero-shot settings. |
| Approach: | They investigate cross-lingual transfer abilities of XLM-R for Chinese and English natural language inference using a large scale Chinese dataset. |
| Outcome: | The proposed model trains on Chinese and English natural language inference datasets. |
Towards Multi-Sense Cross-Lingual Alignment of Contextual Embeddings (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to learn cross-lingual word embeddings are sense agnostic . a novel framework to align contextual embeddables at the sense level is proposed . |
| Approach: | They propose a framework to align contextual embeddings at the sense level by leveraging cross-lingual signal from bilingual dictionaries only. |
| Outcome: | The proposed framework improves word sense disambiguation tasks by leveraging bilingual dictionaries . compared with baseline results, the proposed models achieve 0.52%, 2.09% and 1.29% performance improvements . |
BasahaCorpus: An Expanded Linguistic Resource for Readability Assessment in Central Philippine Languages (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current research on automatic readability assessment (ARA) has focused on improving the performance of models in high-resource languages such as English. |
| Approach: | They propose a hierarchical cross-lingual modeling approach that takes advantage of a language’s placement in the family tree to increase the amount of available training data. |
| Outcome: | The proposed model improves the performance of models in high-resource languages such as English and Hiligaynon, minasbate, Karay-a, and Rinconada. |
ALEXSIS-PT: A New Resource for Portuguese Lexical Simplification (2022.coling-1)
Copied to clipboard
| Challenge: | Lexical simplification (LS) is the task of replacing complex words with simpler alternatives to make texts more accessible to various target populations. |
| Approach: | They propose to use a Brazilian Portuguese multi-candidate dataset to test LS systems. |
| Outcome: | The proposed model outperforms existing models on Brazilian Portuguese and Brazilian newspaper articles. |
Cross-lingual Spoken Language Understanding with Regularized Representation Alignment (2020.emnlp-main)
Copied to clipboard
| Challenge: | despite promising results, current cross-lingual models suffer from imperfect cross-linguistic representation alignments between the source and target languages, which makes the performance sub-optimal. |
| Approach: | They propose a regularization approach to align word-level and sentence-level representations across languages without external resources. |
| Outcome: | The proposed model outperforms state-of-the-art models in few-shot and zero-shot scenarios and achieves comparable performance to supervised training with all training data. |
Translation Artifacts in Cross-lingual Transfer Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing cross-lingual transfer learning techniques involve human and machine translations. |
| Approach: | They propose to use machine translation to translate test set or training set to introduce subtle artifacts that have a notable impact in existing cross-lingual models. |
| Outcome: | The proposed translation process reduces the lexical overlap between the premise and hypothesis by 4.3 and 2.8 points . the proposed translation-test and zero-shot approaches improve on previous work . |
MLQA: Evaluating Cross-lingual Extractive Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Question answering (QA) models have shown rapid progress enabled by the availability of large, high-quality benchmark datasets. |
| Approach: | They present a multi-way aligned extractive QA evaluation benchmark in 7 languages . they evaluate state-of-the-art cross-lingual models and machine-translation-based baselines . |
| Outcome: | The proposed model is based on MLQA, which has over 12K instances in english and 5K in each other language. |
Cross-lingual Emotion Detection (2022.lrec-1)
Copied to clipboard
| Challenge: | Emotion detection is a useful tool for understanding human behavior, but constructing annotated datasets to train models can be expensive. |
| Approach: | They propose to use English as the source language with Arabic and Spanish as target languages to train models for emotion detection in a target language. |
| Outcome: | The proposed approaches surpass state-of-the-art models in Arabic and Spanish by 4% and 5% respectively. |